General refactoring #11
Conversation
Update Analyzer + Improve tool with extra Ping round Info
Changed the name of the file to CID-gen.go to cid_source.go. The command line will be able to take CIDs from a file and publish them to the network.
Options in creating a new config struct
Moved CidSource interface to cid_source file. Added a file cid source Added a bitswap cid source.
Created function find_cid_source which figures out what type of cid will be used.
There was a problem hiding this comment.
hey @FotiosBistas ! Great work! I think that you are on track!
Just left a few comments. Let me know what you think!
struct FileCIDSource now contains the file, the filename and the scanner used to read the file. The idea is that the GetNewCid() function will use the current state of the scanner to read the next CID along with it's contents. I have doubts about using the parse function here to convert the string read from the file to a CID
Added error handling to the read file method.
pkg/hoarder/cid_source.go
Outdated
| @@ -67,9 +71,31 @@ func (bitswap_cid_source *BitswapCIDSource) Type() string { | |||
| return "bitswap" | |||
| } | |||
|
|
|||
| func read_content_from_file() ([]byte, cid.Cid, error) { | |||
| //Reads CID and content from a given file. Starting idea for the file format should be CID CONTENT([]byte array)\n. | |||
| func read_content_from_file(conf *config.Config) ([]byte, cid.Cid, error) { | |||
There was a problem hiding this comment.
I think you could leave the content aside, it's probably a random one. Furthermore, it isn't relevant for the Cid-Hoarder to know the exact bytes of a given CID.
It would also simplify the step for exporting/importing the CIDs
There was a problem hiding this comment.
If I leave out the content I would need to change the interface type:
type CidSource interface {
GetNewCid() ([]byte, cid.Cid, error)
Type() string
}Every time a GetNewCid() is called it returns the CID , it's content and an nil error if all went well. In the latest commit which handles errors it's more clear how I thought about it.
There was a problem hiding this comment.
Yep, although it is never used.
You can return an empty []byte and it won't crash, it will also save you some time parsing the file with the CIDs.
Might happen that you want to track a CID from where you don't know the content yet
There was a problem hiding this comment.
Okay makes sense. What do you think would be best to change the interface or just return a nil type in the place of []byte when reading from a file?
The usage of []byte is here:
// TODO: is it worth keeping the content?
// getRandomContent returns generates an array of random bytes with the given size and the composed CID of the content
func genRandomContent(byteLen int) ([]byte, cid.Cid, error) {
// generate random bytes
content := make([]byte, byteLen)
rand.Read(content)
//TODO do we have to have different CID types?
// configure the type of CID that we want
pref := cid.Prefix{
Version: 1,
Codec: cid.Raw,
MhType: mh.SHA2_256,
MhLength: -1,
}
// get the CID of the content we just generated
contID, err := pref.Sum(content)
if err != nil {
return content, cid.Cid{}, errors.Wrap(err, "composing CID")
}
log.Infof("generated new CID %s", contID.Hash().B58String())
return content, contID, nil
}Yeah I saw that //TODO now.
There was a problem hiding this comment.
What do you think would be best to change the interface or just return a nil type in the place of []byte when reading from a file?
mmmm, I would just return an empty []byte for now, it might be useful in the future
This refers to content that only gets pinged by the CID hoarder
Generate CIDs method will generate CIDs based on the CidSource type
…nto General-refactoring
Motivation
Needed functionality for the command line interface especially to implement the transferring of CIDs from Optimistic Provide to the CID hoarder.
Description
The CID hoarder can generate/publish content from different types of sources. These sources can be a file containing CIDs, random generation of the content and receiving the cids from bitswap protocol.
Tasks
Implement a basic interface to determine the source of the CIDs and the corresponding content (already implemented but can refactor if new source is needed)
Implement generating random CIDs (Already implemented by @cortze )
Implement the reading from JSON file functionality
Implement asynchronous way to add provider records into the hoarder that are published by other sources.
Implement reading from text file functionality
Implement not publishing the CIDs and pinging existing ones in the network. (the most important for the transferring of the CIDs)
Implement the reading from bitswap functionality